Image-capturing hand-held devices of the above type are widely used. For instance, a digital pen is shown in Applicant's published international application WO 01/16691, which is incorporated herein by reference. As described in WO 01/16691, the digital pen may be used together with a product which has a writing surface and a position code provided thereon. The position code encodes a plurality of positions on the surface and may be detected by the digital pen. Thus, information which is being written on the writing surface by means of the digital pen can be electronically recorded by the pen.
In a position-coding product of the above or other types, the code is often represented by a graphical pattern of geometrical objects, marks or symbols, such as empty or filled circles (dots), squares, triangles, etc. The symbols may have uniform or varying appearance (size, color, etc), depending on the actual position coding principle involved. The surface of the product is provided with its position-coding pattern in a suitable way, such as by offset printing, laser printing, inkjet printing or thermal printing.
The digital pen operates by capturing a sequence of images, at a predetermined rate, of the surface of the product as the pen moves across it. In order to decode the position-coding pattern of objects in each captured image, the objects must be safely identified and not be confused with the background of the image. Separating the objects in an image from its background is sometimes referred to as segmentation or binarization. Successful segmentation is facilitated by optimal image quality and, in particular, optimal contrast, i.e. difference in color or greyscale luminance (intensity) between objects and background in the image.
However, image-capturing hand-held devices like the digital pen above are exposed to varying illumination environments due to the very nature of their intended use. For instance, as described in U.S. Pat. No. 5,764,611, a hand-held image-capturing device is typically not held perpendicularly to the surface to be scanned but rather at a varying angle of inclination with respect to the surface. To accurately reproduce the information contained in an image which is captured at an arbitrary inclination, U.S. Pat. No. 5,764,611 proposes use of an inclination sensor in combination with a luminance level detection circuit and a luminance variation calculation circuit. In more detail, the luminance level detection circuit detects average luminance data for each of a predetermined number of sub-areas of an image captured by an image sensor. Each sub-area is low-pass filtered so as to remove effects caused by the objects (black dots) in the image. Thus, the filtered image subareas predominantly contain background data. Average luminance level data are obtained stepwise for the subareas and are accumulated into an average luminance level for the entire image. This average luminance level is used for automatic gain control (AGC) of an image pickup process circuit, which is coupled to the image sensor.
The solution according to U.S. Pat. No. 5,764,611 has a drawback in that it requires complex circuitry in order to perform automatic gain control for the image pickup process circuit. In particular, such automatic gain control will amplify not only the desired portions of the image (i.e., the image signal from the image sensor) but also any noise included therein. Therefore, since noise is inevitable in reality, it is difficult to obtain optimum image contrast by amplifying the image signal.
In addition, many other factors than the inclination angle of the device with respect to the recording surface will contribute to a varying illumination environment for hand-held image-capturing devices. For instance, infrared light emitting diodes (IR LEDs), which are commonly used as illuminating light sources in hand-held image-capturing devices, have a specific irradiance (illumination power), which varies widely between individual diodes. Moreover, light emitting diodes are typically expected to decrease their output power with as much as 50% during their operational lifetime.
Furthermore, external sources of illumination, such as sunlight, may affect the illumination environment. Also, the light scattering or reflecting properties are different for different recording surface materials and may also vary between different parts of the recording surface (e.g. between a paper background and the dots printed thereon).
Consequently, the illumination environment depends on many different parameters, which are difficult to compensate fully and dynamically for by adjusting the image sensor itself or the image processing circuitry, which is coupled to the image sensor. In any event, such compensation would require complex circuitry, which is expensive in terms of processing power as well as component costs.
Generally speaking, digital pens are consumer products and should therefore be manufactured at low cost. Manufacturing cost is increased by either the usage of high-precision data capture components, or the need to trim every manufactured pen to accommodate for variations in its illumination system and/or image detection system, as caused by manufacture or assembly. In a consumer product, it is also undesirable to have the user periodically recalibrate the pen to accommodate for temporal changes of the illumination system. Also, it is objectionable to put severe constraints on the digital pen usage, for example in terms of allowable writing angles, allowable surface materials, and allowable writing environments. This is contrasted by the fact that the digital pen may not be used for writing under such conditions that more than an occasional image in the sequence of images is undecodable, since any resulting errors in the decoding adversely affects the quality of the electronic version of the handwritten information.
The foregoing problem discussion is at least partly applicable to other types of image-capturing devices, which operate under varying illumination conditions and interpret, identify, decode or otherwise read information contained in captured images.
The prior art comprises such devices with exposure control. WO 97/05560 discloses a so-called dataform reader which uses a prestored look-up table to determine a subsequent exposure time, based on a current sensor gain value and a property of a current image. Such a look-up table limits the use of the reader to well-defined, pre-calibrated conditions. U.S. Pat. No. 5,223,935 discloses an electronic still camera with automatic exposure control in which prestored conversion tables are used to set the exposure time. EP-0 602 637 and U.S. Pat. No. 5,831,254 disclose bar code readers which adjust an exposure time control signal to a light source and an image sensor, respectively, by a predetermined amount until a property of the resulting images is found to be acceptable. Whenever used outside a specific operation condition, such bar code readers are likely to exhibit a lengthy adjustment phase or instabilities, leading to a potential loss of information in a considerable number of captured images.